Temporal placement of a rebuffering event

ABSTRACT

A method includes receiving, with a computing system, data representing a video item into a buffer. The method further includes outputting the video item from the buffer to a display system. The method further includes determining that utilization of the buffer falls below a predetermined threshold. The method further includes, in response to determining that the utilization of the buffer falls below the predetermined threshold, determining that there is a specified rebuffering point within a predetermined time frame. The method further includes pausing with the computing system, the video item at the specified rebuffering point in response to determining that there is the specified rebuffering point within the predetermined time frame.

TECHNICAL FIELD

The present disclosure relates generally to video streaming, and moreparticularly, to the timing of a rebuffering event within a videostream.

BACKGROUND

While consumers may access media items, such as movies and televisionshows, by receiving over the air signals or by subscribing to a cable orsatellite television provider, increasingly consumers are accessingcontent over Internet-based systems. Some Internet-based systems allowusers to download or stream content over the Internet to a variety ofclient devices. For example, an Internet-based media system may providecontent to users via a personal computer, a set-top box, or a personalmobile device, such as a smart phone or tablet computer. In particular,streaming media systems enable users to access media content in astreaming video format, such that the users may begin consuming (e.g.,watching and/or listening to) content before the entirety of the contentis delivered to a given user's client device. Such a system allows usersto access content while avoiding a potentially lengthy download processbefore beginning to consume their selected content.

When streaming a media item, the user does not have to download theentire media item before being able to view the media item. Instead, theuser can start consuming the media item almost immediately as soon asthe data representing the media is delivered to the user's device. Asthe data representing the media item is delivered to the user's device,it is temporarily placed in a buffer. The buffer allows for a smootherviewing experience because if the network connection between the user'sdevice and the server from which data is being streamed is temporarilydisrupted or slowed, consumption of the media item may continue usingdata in the buffer. Ideally, the rate at which data is delivered to thebuffer is greater than the rate at which data is read out of the bufferfor display on the client device. However, in some cases, the data isread out of the buffer faster than it is delivered to the buffer. In theevent that the buffer utilization (i.e., the amount of data currentlywithin the buffer) falls to zero, then the media stream is typicallypaused until sufficient data can be delivered to the buffer. This canlower the quality of experience for a viewer, since this abrupt stoppingof playback is perceived by the viewer as a frozen screen—sometimesoverlaid with an hour-glass or spinning-wheel icon—and a correspondingabsence of audio, i.e. silence. The user will generally have noawareness of the amount of data in the buffer, and thus the userperceives the stopping of playback as a randomly-timed event. Whensufficient data is again received into the buffer, playback of the mediaitem may abruptly resume. The resulting discontinuous playback of themedia item is not enjoyable for the user.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing an illustrative server computing system anda client computing system that may be used to perform optimal placementof a rebuffering event, according to some embodiments of the presentdisclosure.

FIG. 2 is a diagram showing specified rebuffering points within a videoitem, according to some embodiments of the present disclosure.

FIGS. 3A, 3B, 3C, and 3D are diagrams showing use of the specifiedrebuffering points for specified rebuffering events, according to someembodiments of the present disclosure.

FIGS. 4A and 4B are tables showing various metadata that may be assignedto portions of a video item to help identify specified rebufferingpoints, according to some embodiments of the present disclosure.

FIG. 5 is a diagram showing detection of features indicating a close-upshot in a portion of a video item, according to some embodiments of thepresent disclosure.

FIG. 6 is a diagram showing detection of features indicating anestablishing shot in a portion of a video item, according to someembodiments of the present disclosure.

FIG. 7 is a diagram showing detection of features indicating a zoom-outshot in a portion of a video item, according to some embodiments of thepresent disclosure.

FIG. 8 is a flowchart showing an illustrative method for pausing a videoitem at a specified rebuffering point, according to some embodiments ofthe present disclosure.

These drawings will be better understood by those of ordinary skill inthe art by reference to the following detailed description.

DETAILED DESCRIPTION

With reference to the drawings briefly described above, exemplaryapplications of systems and methods according to the present disclosureare described in this section. These examples are provided to addcontext and aid in the understanding of the invention. It will thus beapparent to one skilled in the art that the present invention may bepracticed without some or all of these specific details. In otherinstances, well-known process steps or operations have not beendescribed in detail in order to avoid unnecessarily obscuring thepresent disclosure. Additionally, other applications of the concepts andprinciples described herein are possible, such that the followingexamples should not be taken as limiting. The principles and conceptsdescribed herein may be applied to select an optimized time for arebuffering event that may decrease at least some of the negativeaspects of rebuffering for viewers of streamed media items.

In the following detailed description, references are made to theaccompanying drawings, which form a part of the description and in whichare shown, by way of illustration, specific embodiments of the presentdisclosure. Although these embodiments are described in sufficientdetail to enable one skilled in the art to practice the invention, it isunderstood that these examples are not limiting, such that otherembodiments may be used, and changes may be made without departing fromthe spirit and scope of the invention.

As described above, in conventional streaming implementations, if datais not delivered to the client device fast enough, the video must bepaused to wait for more data. After a sufficient amount of data has beenreceived, the video can resume. Various factors, such as networkconditions, may affect the ability of the client device to receive datafast enough. Because there is often little, if any, control over networkconditions, such pauses may appear as random to the user, ofteninterrupting important story elements. These interruptions are typicallyperceived as a frozen screen with no audio. The user may also bepresented with an indication of delay, such as a “Buffering” message, aspinning wheel, or an hourglass icon. These pauses, which allow thebuffer to fill up with additional data, will be referred to asrebuffering events.

Rebuffering events are frustrating to a viewer because they interruptthe viewing experience. Rebuffering may be even more undesirable whenthe video is paused in the middle of an important scene or importantdialogue. Accordingly, the present disclosure relates to methods andsystems that optimize the placement of pauses for necessary rebufferingevents so as to avoid a pause in the video stream at an undesirabletime. In other words, if rebuffering events must occur, then therebuffering points may be set to occur at times that are lessinconvenient for the user instead of at random.

In one example, the locations for rebuffering points may be identifiedwithin a particular piece of content such as a video item. Theseidentified locations will be referred to as specified rebufferingpoints. The specified rebuffering points may correspond to moments ortimes within the video item that are less inconvenient for rebuffering.For example, the specified rebuffering points may correspond to scenechanges, shot changes, or pauses in dialogue. In one example, if bufferutilization gets too low, then the video item may be paused at one ofthe specified rebuffering points, even if the buffer utilization is notyet below the point at which a rebuffering event would otherwise betriggered. In other words, the video stream may be paused forrebuffering before such a pause for rebuffering would occur in aconventional implementation of video streaming where rebuffering occursupon buffer depletion.

Additionally, in some examples, if the network connection from astreaming server to a client system is such that the rate at which datais delivered to the buffer is less than the rate at which data isconsumed, then the video item may be briefly paused at certain specifiedrebuffering points instead of at apparently random points within thevideo due to depletion of data within the buffer.

FIG. 1 is a diagram showing an illustrative server computing system 102and a client computing system 120 that may be used to perform optimalplacement of a rebuffering event. The server computing system 102 may beone of many servers that are owned, operated, or otherwise managed by astreaming media service. The server computing system 102 includes aprocessor 108, a memory 104, a rebuffering point identification module110, and a network interface 112. The server computing system 102 mayfurther include, for example, stand-alone and enterprise-class serversoperating a server operating system (OS) such as a MICROSOFT® OS, aUNIX® OS, a LINUX® OS, or another suitable server-based operating system102. The server computing system 102 may be a server in acontent-distribution network. It should be appreciated that the servercomputing system 102 illustrated in FIG. 1 may be deployed in other waysand that the operations performed and/or the services provided by suchservers may be combined or separated for a given implementation and maybe performed by a greater number or fewer number of individual serverdevices.

The processor 108 may include one or more individual processors orprocessing cores. Similarly, the memory 104 may include one or morememory devices of different types. The memory 104 may storemachine-readable instructions for execution on the processor 108. Suchinstructions may be associated with various applications as well as anoperating system. Additionally, the memory 104 may store video items 106for streaming. Such video items 106 may include, for example,full-length movies, episodes of a series, or portions of a movie orepisode. Other types of video items are also contemplated. The videoitems 106 may be encoded in a machine-readable format that is suited forstorage and/or streaming. More specifically, the video items 106 mayinclude a set of data that specifies a series of frames, each frameincluding an image or information that can be processed to obtain animage. When the frames are displayed in sequence, they produce a video.The video items 106 may be encoded using various technologies thatcompress the data. The data that comprises the video items 106 may alsoinclude audio data corresponding to the video data.

As noted, the memory 104 may include a plurality of memory modules. Thememory modules may be of varying types. Some of the memory 104 may bevolatile memory such as Random Access Memory (RAM). Some of the memory104 may be non-volatile memory such as hard disk drives or Solid StateDrives (SSDs). In addition to storing machine-readable instructions thatform applications, the memory 104 may store video items 106 for analysisand for streaming. The memory 104 may also store the results of suchanalysis, such as the locations or times within the video items 106corresponding to specified rebuffering points.

The server computing system 102 includes a network interface 112 thatallows the server computing system 102 to communicate with othercomputing systems over a network 114 such as the Internet. The networkinterface 112 includes the hardware, software, or combination of both toencode data for transmission over the network 114. The network interface112 may utilize a variety of communication technologies such asEthernet, fiber optic, and wireless communication technologies.

In some examples, a server computing system 102 that stores the videoitems 106 may perform the analysis of the video items 106 to identifyrebuffering points. In such examples, the server computing system 102may include a rebuffering point identification module 110. Therebuffering point identification module 110 includes the hardware,software, or combination of both that analyzes the data representing thevideo items 106 and identifies the placement of rebuffering points. Theidentified rebuffering points may be represented in a variety of ways.In one example, the identified rebuffering points may be represented aspoints in time (e.g., a timestamp including minutes, seconds, andfractions of a second) within the video items 106. In some examples, theframes corresponding to the rebuffering points may be tagged withmetadata that indicates that such frames correspond to rebufferingpoints. The placement of the identified rebuffering points may betransmitted to the client computing system 120 at the outset ofstreaming, such as in a manifest file or in a metadata file pointed toin the manifest file, or during the streaming process. The manifest filemay also point to the location of one or more streams available to auser for the same audiovisual content, the streams having different bitrates and/or qualities. A client computing system 120 with a highquality connection over the network 114 to the server computing system102 may select a high quality stream from among a plurality of streamsidentified in the manifest file. A client computing system 120 with alow quality (e.g., low bandwidth) connection may select a low qualitystream. In the event that the quality of the connection over the network114 changes, the client computing system 120 may begin accessing adifferent stream as appropriate to provide the user with an optimalexperience. The placement of rebuffering points may be indicated in datacontained in the streams, such as in individual frames, or in a separatefile. For example, rebuffering points may be included in SupplementalEnhancement Information (SEI) messages such as are included by H.264codecs or other comparable methods of transmitting supplementalinformation, like captioning or descriptive text, in a data stream.

In some examples, the rebuffering point identification module 110identifies rebuffering points within a video item 106 before that videoitem 106 is streamed. For example, if the video item 106 is only beingstored and not currently being streamed, then the rebuffering pointidentification module 110 may use that time to perform an analysis ofthe video item 106 to determine the rebuffering points. In someembodiments, the analysis may be performed when the video item 106 isbeing encoded from a format optimized for high-fidelity storage to acompressed format suitable for streaming video. In some examples, therebuffering points may be identified before the video item 106 isuploaded to the server computing system 102. In some examples, therebuffering points may be identified by a separate server (not shown)positioned within the network 114 between the server computing system102 that stores the video item 106 and the client computing system 120that consumes the video item 106.

The streaming process involves transferring data representing a selectedone of the video items 106 from the server computing system 102 to aclient computing system 120 through a network 114 such as the Internet.In the present example, the client computing system 120 includes aprocessor 128, a memory 122, a network interface 126, a rebufferingpoint identification module 130, and a display system 132. In someexamples, the client computing system 120 may be, for example, adesktop, laptop, video console, or tablet computing device.

Like the memory 104 of the server 102, the memory 122 of the clientcomputing system 120 may include one or more memory devices of differenttypes. The memory 104 may store machine-readable instructions forexecution on the processor 128. Such instructions may be associated withvarious applications as well as an operating system. Additionally, thememory 122 may include a buffer 124. The buffer 124 is a portion ofmemory used to temporarily store data that is received over the network114 before the data is processed for display to a user 116 via thedisplay system 132. The rate at which data is consumed may depend on theresolution and/or bitrate of the current stream. For example, if theresolution or bit rate of the stream is relatively high, then more datais used to produce the video. Thus, more data is consumed within aparticular period of time in order to present the video item to the user116. Conversely, if the resolution or bitrate of the stream isrelatively low, then less data is used to produce the video. Thus, lessdata is consumed within a particular period of time in order to presentthe video to the user 116. Accordingly, the size of the buffer 124 maybe set or adapted based on the bit rate of the video item streamselected.

In some examples, the client computing system 120 may be responsible forthe identification of rebuffering points within a video item 106. Insuch cases, the client computing system 120 includes a rebuffering pointidentification module 130. In some examples, the rebuffering pointidentification module 130 may be configured to analyze the content 106as it is received in order to identify rebuffering points. In otherwords, rebuffering points are identified in situ on the client computingsystem 120. This may eliminate the need for analysis and metadatageneration associated with rebuffering point identification by theserver computing system 102.

Like the network interface 112 of the server system 102, the networkinterface 126 of the client computing system 120 includes the hardware,software, or combination of both to encode and decode data forcommunications over the network 114. This data is received from theserver system 102 over the network 114 through the network interface126, and the data is placed within the buffer 124.

The display system 132 includes the components used to present videoitems to the user 116. For example, the display system 132 may includevideo and audio processing hardware and software. The display system 132may also include a video output device such as a monitor or screen. Thedisplay system 132 may also include audio output devices such asspeakers or headphones or connections configured to communicate withaudio output devices.

FIG. 2 is a diagram showing specified rebuffering points 202, 204, 206,208. According to the present example, a video item 200 includesrebuffering points 202 corresponding to scene changes, rebufferingpoints 204 corresponding to shot changes, rebuffering points 206corresponding to dialogue pauses, and rebuffering points 208corresponding to images having certain visual characteristics, such aslow brightness. Other criteria may be used to predetermine suitablelocations for rebuffering events.

In some examples, the rebuffering point identification module 110, 130may analyze the data representing the video item 200 and identify eachscene change. A scene generally refers to a section of a film involvinga particular action at a single location and covering a continuousperiod of time. The rebuffering point identification module 110, 130 maydetect scene changes through a variety of different means. In oneexample, the data representing the colors within the image may beanalyzed to determine that an overall color appearance changes from oneframe to the next. For example, an outdoor scene may have a variety ofcolors within the image while a scene change to an indoor scene may havea different variety of colors. Additionally, the data representing theaudio portion may be analyzed to determine that the audio ambience ofthe video has changed, thus suggesting a scene change. For example, ascene in which a character is driving a car may include the sounds ofthe car and traffic. The scene may then change to a scene in a quietroom in which characters are talking to each other. Such change in audioambience may help identify a scene change. Because scene changestypically present a natural shift in the story, they may be lessinconvenient for a rebuffering event. Thus, the frames corresponding toscene changes may be identified as rebuffering points 202.

In some examples, the rebuffering point identification module 110, 130may analyze the data representing the video item 200 and identify eachshot change as described further herein. A shot may generally representa portion of the video that includes one continuous camera roll. A shotmay also generally represent continuous footage between two edits orcuts. The rebuffering point identification module 110, 130 may detectshots in a variety of ways. In one example, the features within eachframe are analyzed. Such features may include the coloring of the frameor the brightness of the frame. Additionally, image processingalgorithms may identify features such as shapes, edges, and corners tobe able to track these features across multiple frames. If a particularframe is substantially different than the previous frame, then it may bedetermined that a cut has occurred. In other words, the previous shothas ended and a new shot has started. Sometimes, shot changes occurduring the middle of dialogue or in the middle of another continuingsound source, such as a jet passing by. In some cases, shot changes donot occur during the middle of the dialogue. Thus, a distinction may bemade between shot changes that occur during dialogue or during acontinuing sound source and shot changes that occur between dialogues orwithout continuing sound sources. A rebuffering event at a shot changethat occurs between dialogues or continuing sound sources may beperceived by the user 116 as less inconvenient. Accordingly, the framescorresponding to such shot changes may be identified as rebufferingpoints 204.

In some examples, the rebuffering point identification module 110, 130may analyze the audio data within the within the video item 200 andidentify pauses in dialogue. Dialogue may be detected by the detectionof audio transients. Various functions can be used to analyze the dataand determine when characters are speaking and when they are not. Insome examples, if the length of a pause in the middle of a dialogue isabove a predefined threshold, then some point of time within that pausemay be identified as a rebuffering point 206.

In some examples, the rebuffering point identification module 110, 130may analyze the data representing the video item 200 for points at whichthe visual properties have specified characteristics such as lowbrightness or relative low brightness. For example, a visually darkerportion of the video item may be defined as any point within the videoitem 200 at which the overall brightness of the image is below anabsolute threshold value or if the brightness of the image below athreshold value based on the preceding image or frame. During suchportions, there may be fewer distinctive features within the video thatare of a particular interest to a viewer. Thus, such visually darkerportions may be less inconvenient for a rebuffering event. Accordingly,such portions may be identified as rebuffering points 208.

In some examples, instead of a single point in time, a single timestamp,or a single frame, within the video item 200 being identified as arebuffering point, a rebuffering time range 210 may be identified. Forexample, it may be determined that a particular time range 210corresponds to a pause in dialogue, or visually darker portion of thevideo item 200. In such cases, any period of time within that time range210 may be considered less inconvenient for a rebuffering event.

In some examples, certain types of rebuffering points may be preferredover others. For example, if possible, it may be more preferable topause for a rebuffering event during a scene change than during a shotchange. Similarly, it may be more preferable to pause for a rebufferingevent during a scene change than during a pause in dialogue. Thus, therebuffering points may be assigned a ranking value. The logic thatdetermines when the rebuffering point should be used for a rebufferingevent may take into account the ranking value of the immediaterebuffering point. For example, if the need for a rebuffering event ismore immediate, then lower ranked rebuffering points such as thosecorresponding to visually darker portions or pauses in dialogue may beused for rebuffering events. If, however, the need for a rebufferingevent is less immediate, then only higher ranked rebuffering points,such as a scene change, may be used for a rebuffering event. In someembodiments, a rebuffering event may occur by default at certainrebuffering points, such as at a dark scene change, i.e. where a scenegoes black or fades to black, accompanied by silence in audio data. Therebuffering may be performed unless the amount of data in the bufferexceeds a certain threshold, such as ten seconds, thirty seconds, or aminute. In such instances, the rebuffering may be performed for a shortamount of time. For example, the rebuffering may be performed for lessthan a second or less than half a second. Including such pauses when theuser 116 is unlikely to perceive the pause may enable the buffer to beexpanded in anticipation of any drops in network connectivity, such aswhen switching between cell towers.

Use of the rebuffering points may vary depending on a variety ofconditions associated with the client computing system 120 receiving thevideo stream. For example, various network conditions may affect use ofthe rebuffering points. If the network connection between the streamingservice provider and the client computing system 120 is slower than theideal rate for streaming at a particular resolution, then the clientcomputing system 120 may be configured to pause at particularrebuffering points in order to keep the buffer utilization above thedesired threshold. For example, the client computing system 120 maypause for a few seconds or for less than a second at each scene changein order to ensure that the buffer utilization does not drop below thedesired threshold. In some examples, if the rate at which content can bedelivered over the network to the client computing system 120 is lower,then the client computing system may pause at lower ranked rebufferingpoints such as pauses in dialogue as well.

In some examples, the length of each pause may depend upon the currentrate at which data for the video item 200 is being delivered to theclient computing system 120. In general, a lower rate may be handled bypausing for longer periods of time at the rebuffering points.Conversely, a higher rate may be handled by pausing for shorter periodsof time at the rebuffering points. The length of such pauses may beadjusted dynamically as network conditions change during the streamingprocess. For example, if network conditions improve, the length of suchpauses may be shortened. In some examples, if network conditions improvesubstantially, it may be determined that no pauses are needed to keepbuffer utilization above the desired threshold. Conversely, if networkconditions degrade, then longer pauses may be applied at the rebufferingpoints. Accordingly, a current rate of rebuffering may be utilized indetermining the lengths of time used to rebuffering as well as the typesof rebuffering points that are used for rebuffering.

The length of the pauses for rebuffering events may also vary dependingon the ranking or type of a particular rebuffering point. For example, ahigher ranked rebuffering point such as one corresponding with a scenechange may have longer pauses as such points are less intrusive to theviewing experience. Conversely, lower ranked rebuffering points such asgaps in dialogue may have shorter pauses as such points may be slightlymore intrusive to the viewing experience. For example, given particularconditions, a pause of one second may be used during a scene changewhile a pause of less than 500 milliseconds may be used during a gap indialogue.

FIGS. 3A-D are diagrams showing use of the specified rebuffering points306 for rebuffering events, according to some embodiments of the presentdisclosure. In some examples, pausing for rebuffering events may betriggered when the buffer utilization 310 falls below a predefinedthreshold 308. Buffer utilization 310 may be defined in a variety ofways. In one example, buffer utilization 310 may be defined by theamount of data within the buffer that is currently unconsumed. In someexamples, buffer utilization 310 may be defined by the amount of playingtime for the video item within the buffer. For example, it may be thecase that the buffer has enough data for one minute and 32 seconds ofplay time.

FIGS. 3A-D illustrate a timeline for a video item 200. Line 302represents the point in time within the video item 200 that is currentlybeing displayed for viewing. Line 304 represents the point in timewithin the video item 200 at which the buffer would be consumed assumingno new data is received. In other words, if no more data were to bedelivered to the buffer, the video item could continue playing until thepoint represented by line 304. Thus, the range in time between line 302and line 304 represents the buffer utilization 310.

FIG. 3A shows that the current buffer utilization 310 is above apredefined threshold. Thus, presentation of the video item will operateas normal. In FIG. 3B, as the video item is played, the bufferutilization 310 falls below the predefined threshold 308. This mayhappen because the rate at which data is delivered to the buffer is lessthan the rate at which the data is processed for presentation in thedisplay system 132. Accordingly, it may be determined that the bufferutilization 310 is projected to fall to a point at which a rebufferingevent is needed. Conventional streaming systems may simply pause atwhichever point it is determined that rebuffering event is needed or atthe point that all data in the buffer has been processed. However,according to principles disclosed herein, a predetermined orpre-identified rebuffering point may be used as the point at which therebuffering event occurs. Because the rebuffering point corresponds to apoint in the video item 200 that is less intrusive, the user 116 mayhave better experience.

FIG. 3C shows that line 302 has reached the rebuffering point 306. Atthis rebuffering point, the video item 200 may be paused to allow forrebuffering. The presentation of the video item 200 in the displaysystem 132 may be stopped temporarily. FIG. 3D illustrates a point intime after the video item 200 has been paused and before it resumes. Asillustrated, the additional time provided by the pause allows more datato be added to the buffer, thus expanding the buffer utilization 310 toa point well above the predefined threshold 308. In some examples, thepause for rebuffering at the rebuffering point 306 may be for a periodof time required to allow the buffer utilization to reach apredetermined size before resuming. In some embodiments, the duration ofthe pause for rebuffering may be determined by subsequent rebufferingpoints. For example, the video item 200 may be paused for rebufferinguntil another rebuffering point or another rebuffering point of acertain type is present in the buffered data. Or a certain amount oftime or data may be added after the subsequent rebuffering as arebuffering pad. For example, a pause for rebuffering may last until thedata associated with the second depicted rebuffering point 306 has beenadded to the buffer or until a threshold amount of additional data afterthe data associated with the second depicted rebuffering point 306 hasbeen added.

In some examples, the client system that is displaying the video mayevaluate the current utilization of the buffer at each specifiedrebuffering point. If, at a particular specified rebuffering point, thebuffer utilization is below a particular threshold, then the video itemmay be paused at that particular specified rebuffering point.Additionally, the current utilization of the buffer may help determinehow long the video item should be paused at that particular rebufferingpoint. For example, if the current buffer utilization is close to thethreshold, then the pause may be relatively short. However, if thecurrent buffer utilization is far below the threshold, then the pausemay be relatively long. In the event that the buffer utilization isabove the threshold, then there may be no need to pause the video itemat that rebuffering point.

FIGS. 4A and 4B are tables showing metadata that may be assigned toshots and frames within a piece of content to help identify rebufferingpoints. In some examples, the rebuffering point identification module110, 130 may include the hardware, software, or combination of both toanalyze the data of a video item 200 and identify different shots andframes within the video item 200. For example, the rebuffering pointidentification module 110, 130 may indicate the starting point andstopping point for each different shot within the video item 200 or foreach different shot within a currently buffered portion of the videoitem 200, depending on the implementation. The rebuffering pointidentification module 110, 130 may also identify a selection of frameswithin the video item 200. The rebuffering point identification module110, 130 may also analyze the images associated with each of the framesto detect various features within the frames. Such features may include,for example the face of characters appearing in the shot or frame. Basedon such features, the rebuffering point identification module 110, 130may assign metadata to the various shots and images detected within thevideo item 200. Such metadata may then be used to assign rebufferingpoints within the video item 200 as well as ranking values to thoserebuffering points. For example, a pause for rebuffering may bepreferable during a time when a first actor is displayed rather than asecond actor.

A frame corresponds to a still image that when combined with otherframes in sequence produces a motion picture. The data that forms thevideo item 200 may describe each frame within the video item 200. Therebuffering point identification module 110, 130 may analyze a pluralityof frames within the video item 200 and assign metadata to each of thoseframes based on features detected within those frames. A series offrames may form a shot. A shot may generally represent a portion of thevideo that includes one continuous camera roll. A shot may alsogenerally represent continuous footage between two edits or cuts. Therebuffering point identification module 110, 130 may detect shots in avariety of ways. In one example, the features within each frame areanalyzed using image processing algorithms. Such features may includethe coloring of the frame, the brightness of the frame, ormachine-identifiable features such as shapes and edges. If a particularframe is substantially different than the previous frame, then it may bedetermined that a cut has occurred. In other words, the previous shothas ended and a new shot has started. After various shots have beenidentified, the rebuffering point identification module 110, 130 mayidentify differences between the features of different frames within theshots. These differences may be used to categorize the shot and assignvarious metadata to the shot.

The rebuffering point identification module 110, 130 also analyzes eachshot in order to assign metadata, including a shot category, to thatshot. This may be done by analyzing the machine-readable data thatrepresents frames within the shot. In one example, the rebuffering pointidentification module 110, 130 selects at least two frames within aparticular shot. The rebuffering point identification module 110, 130analyzes the features found within those frames and identifies thedifferences between the features of those frames. If, for example, thespatial relationships between various features of one frame are largerthan the spatial relationships between various features of the otherframe, it may be determined that the shot is a zoom-out shot. If, forexample, the features of one frame are determined to be those of acharacter's face, and the features of the other frame are alsodetermined to be those of the character's face, it may be determinedthat the shot is a close-up shot if the face occupies more than athreshold amount of the frame. If, for example, it is determined thatthe features of one frame have shifted with respect to the features inthe other frame in a particular manner, it may be determined that theshot is an establishing shot. Other types of shots are contemplated aswell. In some examples, the type of shot in which a rebuffering pointexists may affect the ranking value of that rebuffering point. Forexample, it may be less inconvenient to pause during an establishingshot than a close-up shot.

The rebuffering point identification module 110, 130 may also analyze aselect number of frames within the video item 200. Analyzing the frameswithin the video item 200 may involve examining the machine-readabledata that represents the video item 200. In some examples, every singleframe of the video item may be analyzed. In some examples, however,every X number of frames may be analyzed. In some examples, X may bewithin a range of about 5 to 60. Other values for X are contemplated aswell. The rebuffering point identification module 110, 130 may alsoassign metadata to each frame analyzed.

FIG. 4A shows an illustrative table 400 that includes metadata for aparticular frame, also referred to as frame metadata. The metadata mayindicate the visual properties 401 of the frame, the structuralproperties 403 of the frame, and the temporal properties 405 of theframe. The visual properties 401 include brightness 402, sharpness 404,and contrast 406. Other visual properties may include the colorcomposition of the frame or color temperature. The structural properties403 include frame faces 408 and frame saliency 410. The temporalproperties 405 include frame motion 412 and frame direction 414.

The rebuffering point identification module 110, 130 may assign abrightness value 402 to a frame based on the average brightness valueassociated with each pixel within the frame. Specifically, therebuffering point identification module 110, 130 may examine the datathat represents the frame. That data may define color values for eachpixel within the frame. For example, if the data for the pixel isrepresented in the YUV—also known as YCbCr—color scape, the Y value isrepresentative of the pixel luminance. For example, if the data for thepixel is represented in the RGB color space, then the brightness for aparticular pixel may be defined as the average color value for the pixel(e.g., Br=(R+G+B)/3, where Br is the brightness, R is the red colorvalue, G is the green color value, and B is the blue color value). Othermanners for determining a brightness value are contemplated.

The rebuffering point identification module 110, 130 may assign asharpness value 404 and a contrast value 406 to the frame based on ananalysis of the data that defines the frame. For example, therebuffering point identification module 110, 130 may apply a function todetermine a sharpness value and a contrast value of the frame.Sharpness, sometimes referred to as acutance, is a measure of howstrongly the contrast of an image is perceived. Contrast refers to thecolor differences within the image. Various methods for determining asharpness value and a contrast value based on the data that representsthe frame may be used.

The rebuffering point identification module 110, 130 may also identifyfaces 408 that appear within the shot. For example, it may be determinedbased on an analysis of features within the frame that the frameincludes one or more faces. Various facial recognition functions may beapplied to identify the presence of the faces and then identify theactual faces represented in the data. As an example, the Viola-Jonesalgorithm is one popular choice to detect faces. The faces of thevarious characters within the frame may also be assigned a popularityvalue. This popularity value may be derived in a variety of manners. Inone example, the popularity value is based on a percentage of time inwhich that character appears within the video item. In some examples,external sources may be used to determine the popularity of thecharacter. For example, a popularity value may be predefined by a humanuser. In some examples, an analysis of publicly available informationsuch as webpages and social media may be applied in order to assign apopularity value to a particular character. The popularity of thecharacters within a particular shot or scene may also affect the rankingvalue of rebuffering points within that shot or scene. For example,pausing for rebuffering while showing a more popular character or actormay be preferable.

The rebuffering point identification module 110, 130 may assign saliencydata to a frame. The saliency data may include, for example, a saliencymap. A saliency map identifies the uniqueness of portions of an image.For example, the color uniqueness of a portion (e.g., pixel or set ofadjacent pixels) of an image may be identified with a value. A saliencymap may also include a saliency value assigned to the person or objectof focus within the image. For example, the saliency value may identifyhow much that person or object stands out with respect to the backgroundin which that object or person is placed.

The rebuffering point identification module 110, 130 may also determinetemporal features of the frame. For example, by analyzing the datarepresenting the frame, and adjacent frames, it can be determined that aparticular object or person of focus is moving at a particular speed anddirection relative to other objects or the background of the image. Thisinformation can be determined to assign a frame motion value 412 and aframe direction value 414.

The frame motion value 412 and the frame direction value 414 may be usedto determine a specified rebuffering point. For example, if there is alot of motion within the video item at a particular frame, then this mayindicate that more action is occurring during the scene. Thus, it may bemore inconvenient to cause a rebuffering event at that frame. If,however, there is less motion, then it may be a less inconvenient framefor a rebuffering event. Accordingly, a frame that has a frame motionvalue 412 that is less than a predetermined motion value threshold maybe identified as a specified rebuffering point. In some examples, aframe with a low motion value may have a ranking value that is less thana frame corresponding to a shot change or scene change.

The rebuffering point identification module 110, 130 may also determinerebuffering point ranking value 416 for the frame. The rebuffering pointranking value may be based on a variety of factors associated with theframe or the position of the frame within a shot or scene. For example,the ranking value may range from 0-100, with zero being a bad choice fora rebuffering point and 100 being the ideal choice for a rebufferingpoint. Thus, a frame that is positioned within dialogue and depicts animportant character talking may have a rebuffering point rank value ofzero and the frame corresponding to a fade out or fade in at the end orbeginning of a scene may have a rebuffering point rank value closer to100. Additionally, a frame that is positioned at the end or beginning ofa shot may have a rebuffering point rank value close to 75, for example.The rebuffering point ranking value 416 may also be affected by theother properties 401, 403, 405 of the frame and other considerations asdescribed herein. In some embodiments in which the rebuffering pointidentification module 110 executing on the server computing system 102determines the properties 401, 403, and 405, the client computing system120 utilizes this metadata to produce additional metadata including therebuffering point ranking value 416.

FIG. 4B shows an illustrative table 420 that includes metadata for aparticular shot, also referred to as shot metadata. According to thepresent example, the metadata includes a shot category 422, visualproperties 424, structural properties 426, and temporal properties 428.

The shot category 422 identifies the type of shot. For example, shotcategories may include, but are not limited to, a close-up shot, anestablishing shot, a zoom-out shot, or another category of shotsutilized in the television and film industries. Other types of shotcategories may be defined as well. As described above, a shot categorymay be defined based on feature differences between at least twodifferent frames within the shot.

The visual properties data 424 may include information such asbrightness, sharpness and contrast. These may be represented as averagevalues for the shot. For example, a sample of frames from the shot maybe analyzed and averaged to determine various visual property values.The structural properties data 426 may include structural features ofthe shot such as which characters appear within the shot. The temporalproperties data 428 may indicate the direction, if any, in which theshot is moving or in which an identified object in the shot is moving.The temporal properties data 428 may also indicate the direction anyobjects of focus are moving with respect to the background. Other piecesof information that may be helpful for identifying rebuffering pointsmay be included with the shot metadata.

FIG. 5 is a diagram showing detection of features indicating a close-upshot. FIG. 5 illustrates an image of two different frames 502, 504. Inthe present example, the first frame 502 corresponds to an earlier framewithin the shot and the second frame 504 corresponds to a later framewithin the shot. The frames 502, 504 are analyzed to identify certainfeatures within each frame 502, 504. In some examples, the features maybe identified as primary features and secondary features. In the presentexample, the frames 502, 504 have a primary feature 506 a, 506 b whichis the face of a single character appearing within the shot.Additionally, the frames 502, 504 include secondary features 508 a, 508b, such as a portion of the character's clothing (in this example, thecharacter's tie).

In some examples, various functions may be applied to identify primaryfeatures and secondary features. In general, faces of characters will bedesignated as primary features. Other objects that stand out withrespect to the rest of the background may be designated as secondaryfeatures. If there are no faces within a shot, then other mechanisms canbe used to identify a primary feature. For example, the object thatstands out the most may be designated as the primary feature.Alternatively, no primary feature may be identified and only secondaryfeatures may be identified. In some examples, there may be nodistinction made between primary and secondary features.

The shot category may be assigned by comparing the features between thetwo frames. In FIG. 5, a comparison of the primary feature 506 a fromthe first frame 502 with the corresponding primary feature 506 b fromthe second frame 504 shows that there is little difference in size orposition of the primary feature 506. The trace lines between thefeatures are substantially parallel and horizontal. This indicates thatthere is little motion between the first frame 502 and the second frame504. Additionally, the comparison between the secondary feature 508 afrom the first frame 502 and the secondary feature 508 b from the secondframe 504 shows that there is little difference in position of thesecondary feature 508. Additionally, the primary feature 506 takes up acertain amount of space within the frame. For example, the primaryfeature 506 may have overall dimensions that include at least onedimension that is greater than one third of the corresponding dimensionof the overall frame. For example, the face identified as the primaryfeatures 506 a, 506 b has a height that is greater than one third of theoverall height of the frame. The threshold value of one third isprovided by way of example; other values or percentages may be used inother embodiments. Based on this information, it may be determined thatthe shot is a close-up shot. Thus, the shot may be categorizedaccordingly.

FIG. 6 is a diagram showing detection of features indicating anestablishing shot. FIG. 6 illustrates images of two different frames602, 604. The primary features 606 a, 606 b detected within the framesare people and the secondary features 608 a, 608 b detected within theframes 602, 604 include scenery. The shot category may be assigned bycomparing the features between the two frames 602, 604. A comparison ofthe relation between the primary feature 606 a and the secondary feature608 a of the first frame with the relation between the primary feature606 b and the secondary feature 608 b of the second frame 604 shows thatthe distance between the two changes and that one relative dimensions(for example, height) of the identified features changes while anotherdimension (for example, width) does not change or does not change asmuch. This may indicate that the shot includes vertical movement of thecamera relative to the identified features. In other words, the tracelines between corresponding points within the frames are not completelyhorizontal but are instead slightly diagonal. The relatively shallowslope of the lines indicates that while there is some motion between thetwo frames 502, 504, it is not a sudden or quick motion. Additionally,the primary features 606 a (i.e., the people) take up a relatively smallamount of space compared to the image. Based on this information, it maybe determined that the shot is an establishing shot. Thus, the shot maybe categorized accordingly.

FIG. 7 is a diagram showing detection of features indicating a zoom-outshot. FIG. 7 illustrates images of two different frames 702, 704. Thefeatures 706 a, 706 b detected within the frames include an object offocus at which the character is looking. The shot category may beassigned by comparing the features between the two frames 702, 704. Acomparison of the relative size of the features 706 a in the first frameand the relative size of the features 706 b in the second frame 704shows that the relative size changes. Specifically, the features 706 bwithin the second frame 704 are smaller than the corresponding features706 a of the first frame 702. The converging nature of the trace linesbetween corresponding points suggests that the corresponding featuresare smaller in the second frame 704 than they are in the first frame702. Based on this information, it may be determined that the shot is azoom-out shot. Thus, the shot may be categorized accordingly. If it hadbeen determined that the features 706 b within the second frame 704 werelarger than the corresponding features 706 a of the first frame 702,then it may have been determined that the shot is a zoom-in shot.

FIGS. 5-7 illustrate a few examples of detecting features within shotsto assign a shot category to such shots. Other types of shots may bedetected as well. Additionally, other types of functions for identifyingdifferent types of shots may be used in some embodiments of therebuffering point identification modules described herein.

FIG. 8 is a flowchart showing an illustrative method for pausing videoitem at specified rebuffering points. The method 800 includes severalenumerated steps or operations. Embodiments of the method 800 mayinclude additional steps before, after, in between, or as part of theenumerated steps. Some embodiments may omit one or more of theenumerated operations. Additionally, some embodiments of the method 800include non-transitory machine-readable media having instructions storedthereon that cause a processor to perform all or some of the describedoperations. According to the present example, the method 800 includes anoperation 802 for receiving a streaming video item into a buffer orextracting or reading data that comprises the streaming video item. Thevideo item may be, for example, a full-length movie or an episode of aseries. Other examples of video items are contemplated as well. Thevideo item may be delivered as a stream. In other words, the video itemis displayed by a client device while data is still being transferred tothe client device from a server device. Thus, the video item can beginpresentation before it is entirely downloaded to the client device.

The method 800 further includes an operation 804 for outputting thevideo item from the buffer to a display system. The display system mayinclude, for example, a visual output device such as a screen and anaudio output device, such as a set of speakers or a connection thereto.Outputting the video item from the buffer to the display system consumesthe data within the buffer. When viewing streaming video items in such amanner, data representing a video item is received from the remoteserver into a local buffer. Conventionally, if such data within thebuffer is consumed for presentation before additional data can bedelivered, the video item has to pause to allow for rebuffering. Suchrebuffering events do not take into account aspects of the content beingrendered and can reduce the quality of experience by the viewer.

The method 800 further includes an operation 806 for determining thatthere is a specified rebuffering point within a predetermined timeframe. The specified rebuffering point may correspond to a scene change,a shot change, or other point within the video item that is lessinconvenient for a pause. The rebuffering points may be identifiedbefore the video item is streamed or may be identified by the clientcomputing system as data is put into the buffer. The predetermined timeframe may correspond to the data that is currently within the buffer.Specifically, the predetermined time frame may extend between the pointin which the video is currently at and the point at which the buffer isentirely consumed. In some examples, there may be no specifiedrebuffering point within the unconsumed data within the buffer. However,it may be the case that such a rebuffering point will appear before thebuffer is entirely consumed or a rebuffering event is otherwisetriggered. Embodiments of the method 800 may include an operation ofselecting a particular rebuffering point from among a plurality ofpotential rebuffering points, in response to determining that the bufferutilization falls below a predetermined threshold. The bufferutilization may be determined to fall below the predetermined thresholdwhen the amount of data in the buffer falls below a predetermined datavalue or when the amount of data in the buffer is decreasing at a ratepredictive of falling below a predetermined threshold within apredetermined amount of time. Accordingly, the determination may be anactual determination or a predictive determination. Additionally, someembodiments of the method 800 may include an operation of evaluating thebuffer to determine a lever of buffer fullness or what percentage ofmemory allocated to the buffer is actually occupied as in order todetermine whether a pause for rebuffering is necessary. In someembodiments, the evaluation of buffer fullness may be used to determinehow long a pause for rebuffering should be or when a pause should bestopped.

The method 800 further includes an operation 808 for pausing the videoitem at the specified rebuffering point, in response to determining thatthere is a specified rebuffering point within the predetermined timeframe and that rebuffering should be performed to avoid completelyemptying the buffer. By pausing the video item at the specifiedrebuffering point instead of a random point determined only by theamount of data in the buffer, the inconvenience of the rebuffering eventis lessened. Thus, a viewer is provided with a better experience whileviewing the streaming content. In some instances, a rebuffering may beperformed without the viewer perceiving the pause for rebuffering.Accordingly, while conventional implementations of rebuffering eventsgenerally include some alteration of the scene presented to the user tocommunicate that rebuffering is being performed (e.g., a spinning wheelappears over a frame), some embodiments of the present disclosure maynot include any indication to the user that a rebuffering event isoccurring. Because the rebuffering is performed at an identified,appropriate time the rebuffering is less noticeable to the users. Forexample, adding a single second showing a black frame at an identifiedfade out may not be perceived by the user as rebuffering, but simply asa longer transition in the video item.

Additionally, some embodiments of the client computing system 120 may“pause” the playback of the video item by stretching video and/or audiocontent that is present in the buffer. The client computing system 120may interpolate video and audio information so that a given portion ofvideo is stretched to increase in time. For example, the buffer 124 mayinclude data that can be processed into five seconds of video. Some orall of this data can be “stretched” by adding additional frames or bydisplaying frames for longer periods of time to cause the data to bepresented over six seconds, rather than five. The audio may becorrespondingly stretched in a way that does not alter the pitch. Insome embodiments, a Fourier spectrum of the relevant portion of thebuffer audio may be used to create a spectral profile to synthesizesimilar audio content. For example, an establishing shot identified in amedia item may be stretched from ten seconds to twelve seconds to permittwo seconds of rebuffering to occur while the establishing shot isshown. The client computing system 120 may identify the establishingshot by metadata including a tag that indicates a rebuffering pointassociated with the shot. Accordingly, the client computing system 120may provide a rebuffering response based on the type of rebufferingpoint being used for rebuffering.

In some embodiments, after a rebuffering point has been selected, theclient computing system 120 may apply a fade-out to the data that is yetto be displayed by the display system 132. The fade-out may be providedto video and/or audio included in the buffered data. In someembodiments, when the rebuffering is likely to take a significant amountfor time (e.g., more than 3, 5, or 10 seconds) a predetermined audiocontent may be presented. Similarly, a predetermined video content maybe presented, such as a frame showing the title of the video item or aframe associated with a chapter or segment of the video item. Whenpredetermined audio content is presented, the content may be designedfor looping and may be obtained from the video item itself, such as asoundtrack of the video item. Such approaches may provide for a betteruser experience during extending rebuffering periods and may be includedin embodiments of the method 800.

Some examples of processing systems described herein may includenon-transient, tangible, machine readable media that include executablecode that when run by one or more processors may cause the one or moreprocessors to perform the operations of method 800 as described above.Some common forms of machine readable media that may include theoperations of method 800 are, for example, floppy disk, flexible disk,hard disk, magnetic tape, any other magnetic medium, CD-ROM, any otheroptical medium, punch cards, paper tape, any other physical medium withpatterns of holes, RAM, PROM, EPROM, FLASH-EPROM, any other memory chipor cartridge, and/or any other medium from which a processor or computeris adapted to read.

Embodiments of the present disclosure may improve server and clientsystems utilized in a streaming media environment by optimizing theplacement of rebuffering events. The optimized placement may bedetermined based on the data being streamed, the content of the databeing streamed, and network conditions experienced by the client device.The optimized placement may improve these computing systems so as toprovide for an improved user experience.

Although illustrative embodiments have been shown and described, a widerange of modification, change and substitution is contemplated in theforegoing disclosure and in some instances, some features of theembodiments may be employed without a corresponding use of otherfeatures. One of ordinary skill in the art would recognize manyvariations, alternatives, and modifications. The claims should beconstrued broadly and in a manner consistent with the scope of theembodiments disclosed herein. Accordingly, certain aspects of thepresent disclosure are set out the following numbered clauses:

1. A method comprising: receiving, with a computing system, datarepresenting a video item into a buffer; outputting, with the computingsystem, the video item from the buffer to a display system; determining,with the computing system that utilization of the buffer falls below apredetermined threshold; in response to determining that the utilizationof the buffer falls below the predetermined threshold, determining thatthere is a specified rebuffering point within a predetermined timeframe; and pausing with the computing system, the video item at thespecified rebuffering point in response to determining that there is thespecified rebuffering point within the predetermined time frame.

2. The method of clause 1, further comprising resuming the video itemafter a predetermined amount of time, the predetermined amount of timebeing based on a rate at which the video item is received by the buffer.

3. The method of any of clauses 1-2, wherein the specified rebufferingpoint corresponds to one of: a shot change and a scene change.

4. The method of any of clauses 1-3 wherein the specified rebufferingpoint corresponds to a in which a frame motion value is below apredetermined motion value threshold.

5. The method of any of clauses 1-4, wherein the specified rebufferingpoint corresponds to a point at which values representing visualproperties are below a predefined visual property threshold.

6. The method of any of clauses 1-5, wherein the specified rebufferingpoint is a pause in a dialogue of the video item.

7. The method of any of clauses 1-6, wherein the specified rebufferingpoint is selected from among a plurality of potential rebuffering pointswithin the video item, the plurality of potential rebuffering pointshaving different types, the different types being assigned differentranking values.

8. The method of clause 7, wherein the pausing the video item occursonly if the specified rebuffering point is above a specific rank.

9. The method of any of clauses 1-8, wherein the specified rebufferingpoint is identified within the video item before the video item isstreamed to the computing system.

10. The method of any of clauses 1-9, wherein the specified rebufferingpoint is identified by the computing system while a portion of the videoitem in which the specified rebuffering point is positioned is withinthe buffer.

11. A method comprising: receiving, with a computing system, first datarepresenting a video item into a buffer and second data indicating a setof specified rebuffering points for the video item; outputting, with thecomputing system, the first data representing the video item from thebuffer to a display system; determining that a rate at which the firstdata is received is less than a rate at which the first data is outputfrom the buffer to the display system by a threshold amount; and inresponse to determining that the rate at which the first data isreceived is less than the rate at which the first data is output to thedisplay system by the threshold amount, pausing the media stream at oneof the set of specified rebuffering points.

12. The method of clause 11, wherein the set of specified rebufferingpoints includes shot changes, scene changes, scenes with predefinedvisual characteristics, and pauses in dialogue.

13. The method of clause 12, wherein different types of rebufferingpoints are assigned different rankings.

14. The method of clause 13, wherein the one of the set of specifiedrebuffering points is selected based on a difference the rate at whichthe first data is received and the rate at which the first data isoutput from the buffer to the display system.

15. The method of clause 14, wherein a subset of acceptable rebufferingpoints from which the selected rebuffering point is selected includesadditional types of rebuffering points as the difference increases.

16. The method of any of clauses 13-15, further comprising pausing themedia stream at a plurality of rebuffering points of the set ofrebuffering points, wherein a pause at each of the plurality of selectedrebuffering points is for a different period of time based on a rankingvalue assigned to that rebuffering point.

17. The method of any of clauses 11-16, wherein a period of time forwhich the video item is paused is based on a prediction of when thebuffer will be below a predetermined value.

18. A method of optimizing placement of a rebuffering event in steamingmedia playback, the method comprising: receiving, with a servercomputing system, a streaming video item; performing image processing toidentify a plurality of specified rebuffering points in the steamingvideo item; storing, in a memory, information characterizing theidentified plurality of specified rebuffering points; and transmittingthe information characterizing the identified plurality of specifiedrebuffering points over a network to a client computing system.

19. The method of claim 18, wherein the information characterizing theidentified plurality of specified rebuffering points includes at leastone of: timestamps and frame numbers associated with a specifiedrebuffering point and a tag indicating a rebuffering point typeassociated with each specified rebuffering point.

20. The method of claim 18, further comprising transmitting thestreaming video item to the client computing system, whereintransmitting the streaming video item is performed in connection withthe transmitting the information characterizing the identified pluralityof specified rebuffering points.

What is claimed is:
 1. A method comprising: receiving, with a computingsystem, data representing a video item into a buffer; outputting, withthe computing system, the video item from the buffer to a displaysystem; determining, with the computing system, that utilization of thebuffer falls below a predetermined threshold; in response to determiningthat the utilization of the buffer falls below the predeterminedthreshold, determining, with the computing system, that there is aspecified rebuffering point within a predetermined time frame, thespecified rebuffering point being selected from among a plurality ofpotential rebuffering points within the video item, the plurality ofpotential rebuffering points comprising different types of rebufferingpoints that are assigned different ranking values, such that the highestranked rebuffering point is selected as the specified rebuffering point;and pausing, with the computing system, the video item at the specifiedrebuffering point in response to determining that there is the specifiedrebuffering point within the predetermined time frame.
 2. The method ofclaim 1, further comprising resuming the video item after apredetermined amount of time, the predetermined amount of time beingbased on a rate at which the video item is received by the buffer. 3.The method of claim 1, wherein the specified rebuffering pointcorresponds to one of: a shot change and a scene change.
 4. The methodof claim 1, wherein the specified rebuffering point corresponds to a inwhich a frame motion value is below a predetermined motion valuethreshold.
 5. The method of claim 1, wherein the specified rebufferingpoint corresponds to a point at which values representing visualproperties are below a predefined visual property threshold.
 6. Themethod of claim 1, wherein the specified rebuffering point is a pause ina dialogue of the video item.
 7. The method of claim 6, wherein thepausing the video item occurs only if the specified rebuffering point isabove a specific rank.
 8. The method of claim 1, wherein the specifiedrebuffering point is identified within the video item before the videoitem is streamed to the computing system.
 9. The method of claim 1,wherein the specified rebuffering point is identified by the computingsystem while a portion of the video item in which the specifiedrebuffering point is positioned is within the buffer.
 10. A methodcomprising: receiving, with a computing system, first data representinga video item into a buffer and second data indicating a set of specifiedrebuffering points for the video item; outputting, with the computingsystem, the first data representing the video item from the buffer to adisplay system; determining that a rate at which the first data isreceived is less than a rate at which the first data is output from thebuffer to the display system by a threshold amount; and in response todetermining that the rate at which the first data is received is lessthan the rate at which the first data is output to the display system bythe threshold amount, pausing the media stream at a rebuffering pointselected from the set of specified rebuffering points, the set ofpotential rebuffering points comprising different types of rebufferingpoints that are assigned different ranking values, such that the highestranked rebuffering point is used as the selected rebuffering point. 11.The method of claim 10, wherein the set of specified rebuffering pointsincludes shot changes, scene changes, scenes with predefined visualcharacteristics, and pauses in dialogue.
 12. The method of claim 10,wherein a period of time for which the video item is paused is based ona prediction of when the buffer will be below a predetermined value. 13.The method of claim 11, wherein different types of rebuffering pointsare assigned different rankings.
 14. The method of claim 13, wherein theone of the set of specified rebuffering points is selected based on adifference the rate at which the first data is received and the rate atwhich the first data is output from the buffer to the display system.15. The method of claim 13, further comprising pausing the media streamat a plurality of rebuffering points of the set of rebuffering points,wherein a pause at each of the plurality of selected rebuffering pointsis for a different period of time based on a ranking value assigned tothat rebuffering point.
 16. The method of claim 14, wherein a subset ofacceptable rebuffering points from which the selected rebuffering pointis selected includes additional types of rebuffering points as thedifference increases.
 17. A method of optimizing placement of arebuffering event in steaming media playback, the method comprising:receiving, with a server computing system, a streaming video item;performing image processing to identify a plurality of specifiedrebuffering points in the steaming video item, the plurality ofspecified rebuffering points comprising different types of rebufferingpoints that are assigned different ranking values; storing, in a memory,information characterizing the identified plurality of specifiedrebuffering points; and transmitting the information characterizing theidentified plurality of specified rebuffering points over a network to aclient computing system, such that the streaming video item isrebuffered at a selected highest ranked rebuffering point.
 18. Themethod of claim 17, wherein the information characterizing theidentified plurality of specified rebuffering points includes at leastone of: timestamps and frame numbers associated with a specifiedrebuffering point and a tag indicating a rebuffering point typeassociated with each specified rebuffering point.
 19. The method ofclaim 17, further comprising transmitting the streaming video item tothe client computing system, wherein transmitting the streaming videoitem is performed in connection with the transmitting the informationcharacterizing the identified plurality of specified rebuffering points.